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Abstract 

In this paper we give an explicit solution to the rank constrained matrix 
approximation in Frobcnius norm, which is a generalization of the classical 
approximation of an mxn matrix A by a matrix of rank k at most. 
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1 Introduction 

Let C mxn be set of mxn complex valued matrices, and denote by lZ(m, n, k) C C 
the variety of all m x n matrices of rank k at most. Fix A = [c%']™ ; =i € 



Then A* £ C nxm is the conjugate transpose of A, and [|^4||f := \jYlTj=i \ a ij\ 2 
is the Frobenius norm of A. Recall that the singular value decomposition of A, 
abbreviated here as SVD, is given by A = Ua^aVX, where U A € C mxm , V A € C nxn 
are unitary matrices, Yja := diag((Ti(yl), . . . , 0min(m,n) 

(A)) € C mxn is a generalized 
diagonal matrix, with the singular values o~i(A) > 02 (^4) > . . . > on the main 
diagonal. The number of positive singular values of A is r, which is equal to the 
rank of A, denoted by rank A. Let U A = [ui U2 ... u m ], V A = [vi V2 ... v n ] be the 
representations of U, V in terms of their m, n columns respectively. Then Uj and v, 
are called the left and the right singular vectors of A, respectively, that correspond 
to the singular value (Ti(A). Let 

rank A rank A 

P A ,L := € Cm><m ' Pa >R : = E € Cn><n ' ( L1 ) 

i=l i=l 

be the orthogonal projections on the range of A and A* respectively. Denote by 

k 

A k :=Y J <A)u i w*eC mxn 

i=i 

for k = 1, . . . , rank A. For k > rank A we define A^ := A (= ^4 ra nk/0- For 
1 < k < rank A, the matrix A k is uniquely defined if and only if ak(A) > ak+i(A). 
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The enormous application of SVD decomposition of A in pure and applied math- 
ematics, is derived from the following approximation property: 

min \\A-X\\ F = \\A- A k \\ F , k = 1, . . . (1.2) 

X£ll{m,n,k) 

The latter is known as the Eckart- Young theorem [2]. We note that the work 
[2] implied a number of extensions. We cite [4, 5, 7, 8] as some recent refer- 
ences. Another application of SVD is a formula for the Moore-Penrose inverse 
At := V A ^ A U* A e C nxm of A, where 

4 : = dia g(^, • • • > ^TIW' ' • • • «°) e CnXm See fOT ™P le t 1 ]- 

2 Main Result 

Below, we provide generalizations of the classical minimal problem given in (1.2). 

Theorem 2.1 Let matrices A £ C rnxn , B € C mxp and C £ C qxn be given. 
Then 

X = B\P B , L AP c ,B)kC ] (2.1) 
is a solution to the minimal problem 

min \\A-BXC\\f, (2.2) 

X&TL{p,q,k) 

having the minimal \ \X\\p- This solution is unique if and only if either 
k > rank P b ,lAP c ,r or 1 < k < rank P b ,lAP c ,r 

and 

o- k (PB,LA p c,n) > o- k+1 (P B ,LAP c ,R). 

Proof of Theorem 2.1 Recall that the Frobenius norm is invariant under the 
multiplication from the left and the right by the corresponding unitary matrices. 
Hence \\A - BXC\\ F = \\A - S B XS C ||, where A := U* B AV C and X := V£XU C . 
Clearly, X and X have the same rank and the same Frobenius norm. Thus, it is 
enough to consider the minimal problem min \\A — 'E b XT i c\\f- 

XGll(p,q,k) 

Let s = rank B and t = rank C. Clearly if B or C is a zero matrix, then X = 
is the solution to the minimal problem (2.2). In this case either Pb,l ° r Pc,R are 
zero matrices, and the theorem holds trivially in this case. 

Let us consider the case 1 < s,l < t. Define B\ := diag (o~i(B ),..., a s (B)) € 
C sxs ,Ci := diag(ai(C),...,CTi(C)) e C tx *. Partition A and X into four block 
matrices Ay and X^ with i,j = 1,2 so that A = [Aij]f j =1 and X = [Xij]fj =1 , 

where An, Xn £ C sxt . (For certain values of s and t, we may have to partition A 
or X to less than four block matrices.) Next, observe that Z := T, B XT,c = [Zij]fj = i, 
where Z\\ = BiXnCi and all other blocks Zij are zero matrices. Since B\ and C\ 
are invertible we deduce 

rank Z = rank Z\\ = rank Xn < rank X < k. 
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The approximation property of (Au) k yields the inequality \\Au — Zn\\p > \\Au — 
{An)k\\F f° r any Zn of rank k at most. Hence for any Z of the above form, 

\\A-Z\\ F = \\A 11 -Z 11 \\ 2 F + Yl \\A i3 \\ 2 F >\\A u -(A 11 )k\\ 2 F+ £ II^Hf- 

2<i+j<4 2<i+j<4 

Thus X = [X i:j }l j=1 , where X n = Bf 1 (A 11 ) fc Cf 1 and X {j = for all ^ (1, 1) 
is a solution to the problem min \\A — Yi B XYic\\f with the minimal Frobe- 

Xe1Z(p,q,k) 

nius form. This solution is unique if and only if the solution Z\\ = (An)), is the 
unique solution to the problem min ll-Aii — Zu\\f. This happens if either 

Zn67J(s,t,fe) 

k > rank ^4 ii or 1 < A; < rank An and ^(An) > ff^i(Au). A straightforward 
calculation shows that X = Y^ B {P^ B> i,APY, Cl R)k^c- Thus, a solution of (2.2) with 
the minimal Frobenius norm is given by 

X = B<<U B (Px B , L U* B AVcPz c ,R)kVStf 
= Blu B (U B P B>L APc, R Vc)kV3<y 
= B\P B , L AP C) R)kC ] . 

This solution is unique if and only if either k > rank P B ,lAPc,r or 1 < k < 
rank P b ,lAP c ,r and a k (P B , L APc,R) > a k+1 {P B ,LAP c ,R). □ 

A special case of the minimal problem (2.2), where X is a rank one matrix and 
C the identity matrix, was considered by Michael Elad [3] in the context of image 
processing. 



3 Examples 

First observe that the classical approximation problem given by (1.2) is equivalent 
to the case m = p,n = q,B = I m ,C = I n . (Here, I m is the m x m identity matrix.) 
Clearly P/ m ,z, = I m , Pi n ,R = In, Im = Im, A = I n - In this case we obtain the 
classical solution B*(P B> LAPc,R)kC = A k . 

Second, if p = m, q = n and B, C are non-singular, then rank (BXC) = rank X. 
In this case, P Bj l = Im and Pc,R = In, and the solution to (2.2) is given by 
X = B- l A h C- 1 '. 

Next, a particular case of the problem (2.2) occurs in study of a random vector 
estimation (see, for example, [9, 6]) as follows. Let (fi, be a probability space, 
where f2 is the set of outcomes, E a a-field of measurable subsets A C and 
/i : S h [0,1] an associated probability measure on S with /x(O) = 1. Suppose 
that x € L 2 (r2,M m ) and y € L 2 (0,M n ) are random vectors such that x = (x±, . . . , 
x m ) T and y = (yi, . . . , y n ) T with Xi,yj G L 2 (0,]R) for i = l,...,m and j = 1, 
... ,n, respectively. Let E xy = [eij tX y] £ W nxn ,E yy = [ejk, yy ] £ ffi' ixri be correlation 
matrices with entries 

eij,xy = / Xi(u)yj(uj)dn(uj), e jktyy = / yj(uj)y k (uj)diJ,(Lu), 
Jn Jn 

i = 1, . . . , m, j, k = 1, . . . , n, w£(l. 
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The problems considered in [9, 6] are reduced to finding a solution to the problem 
(2.2) with A = E xy Eyy 2 ^ , B = I n and C = Eyy 2 where we write E y !y 2 ^ = {E y !y 2 )^ . 
Let the SVD of Eyy be given by Eyly 2 = V n T,V* and let rank Ely 2 = r. Here, 
v i7---7 v n] with Vj the i-th column of V n . By Theorem 2.1, the solution 
to this particular case of the problem (2.2) having the minimal Frobenius norm is 
given by X = (E X yE X y l2 ^V T V*) k E\ l2 \ where E^V r V* = Eyj? . Therefore, X = 
{E X yE\Jy 2 ^^kE\jy 2 ^ . The conditions for the uniqueness follow directly from Theorem 
2.1. 
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